<html>
<head><meta charset="utf-8"><title>Unit tests categorization · general · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/index.html">general</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Unit.20tests.20categorization.html">Unit tests categorization</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="237387825"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Unit%20tests%20categorization/near/237387825" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Edward <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Unit.20tests.20categorization.html#237387825">(May 04 2021 at 19:52)</a>:</h4>
<p>I'm not sure if this is an appropriate place to ask this question, but anyway. </p>
<p>I haven't found a <em>reasonable</em> way to categorize unit tests. As far as I understood, everything that is annotated by <code>#[test]</code> is getting collected by a compiler and passed into <code>libtest</code>'s runner. <br>
While that's great state of things, one would like to have a bit more control, for example for long-running tests that shouldn't always run. Yes, there's <code>#[ignore]</code>, but that allows only to separate tests into two categories: regular test and ignored test.<br>
What I want:</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="cp">#[test]</span><span class="w"></span>
<span class="cp">#[category(</span><span class="s">"regression"</span><span class="cp">)]</span><span class="w"></span>
<span class="cp">#[issue(</span><span class="s">"#12345"</span><span class="cp">)]</span><span class="w"></span>
<span class="k">fn</span> <span class="nf">test_long_compute</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">   </span><span class="c1">// something that takes a long time</span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<p>Is there a way to achieve that? I know that I can write an external script that would run <code>cargo test $args</code> with specific test cases and mark most of them ignored, but that's definitely less than ideal. <br>
Also, I know there's <a href="https://github.com/rust-lang/rust/issues/50297">https://github.com/rust-lang/rust/issues/50297</a> but not sure if it's that relevant there.</p>



<a name="237391077"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Unit%20tests%20categorization/near/237391077" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Unit.20tests.20categorization.html#237391077">(May 04 2021 at 20:15)</a>:</h4>
<p>you can put them into different modules depending on the category and name the module after the category</p>



<a name="237462717"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Unit%20tests%20categorization/near/237462717" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Edward <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Unit.20tests.20categorization.html#237462717">(May 05 2021 at 09:00)</a>:</h4>
<p>Is there a way to alter how compiler does collect the tests? I.e to make example above work? I know a lot of workarounds and it's not a big painpoint at all, but in the end it's a workaround anyway.</p>



<a name="237474142"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Unit%20tests%20categorization/near/237474142" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Unit.20tests.20categorization.html#237474142">(May 05 2021 at 10:44)</a>:</h4>
<p>Yeah, I want to fix the build issue at some point but it's tricky because right now x.py doesn't really have a concept of individual crates, only "steps"</p>



<a name="237474149"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Unit%20tests%20categorization/near/237474149" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Unit.20tests.20categorization.html#237474149">(May 05 2021 at 10:44)</a>:</h4>
<p>So it's a pretty big change</p>



<a name="237474179"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Unit%20tests%20categorization/near/237474179" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Unit.20tests.20categorization.html#237474179">(May 05 2021 at 10:45)</a>:</h4>
<p>Probably not relevant to you though <span aria-label="laughing" class="emoji emoji-1f606" role="img" title="laughing">:laughing:</span></p>



<a name="237530123"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Unit%20tests%20categorization/near/237530123" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Edward <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Unit.20tests.20categorization.html#237530123">(May 05 2021 at 17:00)</a>:</h4>
<p>Just to clarify, I was talking about altering the rules for aggregating tests from crate, which happens <a href="https://github.com/rust-lang/rust/blob/673d0db5e393e9c64897005b470bfeb6d5aec61b/compiler/rustc_builtin_macros/src/test_harness.rs#L95">~here</a>.<br>
I just realized that I probably could just use <a href="https://doc.rust-lang.org/unstable-book/language-features/custom-test-frameworks.html">https://doc.rust-lang.org/unstable-book/language-features/custom-test-frameworks.html</a> to toy with that :)</p>



<a name="237530375"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Unit%20tests%20categorization/near/237530375" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Unit.20tests.20categorization.html#237530375">(May 05 2021 at 17:02)</a>:</h4>
<p>for that I would just use integration tests so you can use <code>--test</code></p>



<a name="237560601"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Unit%20tests%20categorization/near/237560601" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Unit.20tests.20categorization.html#237560601">(May 05 2021 at 20:33)</a>:</h4>
<p>I'm (very slowly, very early) writing my own dang test framework — <a href="https://github.com/shepmaster/crispy-sniffle/blob/">https://github.com/shepmaster/crispy-sniffle/blob/</a>. It has the ability to skip tests and labels are totally a reasonable thing to add.</p>



<a name="237560848"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Unit%20tests%20categorization/near/237560848" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Unit.20tests.20categorization.html#237560848">(May 05 2021 at 20:34)</a>:</h4>
<p>It looks like this in usage:</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="cp">#[cfg(test)]</span><span class="w"></span>
<span class="k">mod</span> <span class="nn">tests</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="cp">#![tf_macros::foo]</span><span class="w"></span>

<span class="w">    </span><span class="cp">#[before]</span><span class="w"></span>
<span class="w">    </span><span class="k">fn</span> <span class="nf">one</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="fm">println!</span><span class="p">(</span><span class="s">"Setup"</span><span class="p">);</span><span class="w"></span>
<span class="w">    </span><span class="p">}</span><span class="w"></span>

<span class="w">    </span><span class="cp">#[test]</span><span class="w"></span>
<span class="w">    </span><span class="k">fn</span> <span class="nf">use_it</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>

<span class="w">    </span><span class="p">}</span><span class="w"></span>

<span class="w">    </span><span class="k">mod</span> <span class="nn">alpha</span><span class="p">;</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>